Why Feedback Matters

Feedback is often underestimated. As the task gets more complex, feedback becomes more and more important.

Did you realize that at one point all phone calls had to be placed via an operator? That’s right. At one point in time phones didn’t even have dials. You’d pick up the receiver and wait for the operator to ask you which number you wanted to dial and they’d dial it for you. Over time, the phone company realized it’d be cheaper to have customers dial their own number so they didn’t have to employ operators to do something that customers could do for themselves.

But while it’s hard to imagine a situation where we all have to rely on operators to complete phone calls for us, it’s easier to understand how we could eliminate that job. When you place a phone call, you get feedback. If you call a number and it doesn’t ring but instead you get a “that number has been disconnected” or “that number is no longer in service” or some such message then you can easily deduce that either you dialed wrong or the number you were given was wrong. Simple.

If you dial and someone answers the phone you can ask “Have I reached whomever?” and they can tell you yes or no. Simple—easy feedback.

Now also consider that at one point all elevators in buildings employed elevator operators. These people were solely there to run the elevator up or down for people who wished to use the elevator. Of course that occupation (like buggy whip craftsman) is obsolete these days. You get into an elevator, press a button and when the elevator stops you can see if you’re at the floor you want to be on or you aren’t.

Now consider software. I write a program and at the end of running it outputs “42” to the terminal. Did it work correctly? If the program was supposed to add 21 and 21 then yes it worked correctly. If you walk up to the terminal and ask it to run my program—having never seen the source and having no idea of what it does and you see “42” how can you tell if it’s right or not? You’ve got feedback but you have no way of evaluating the correctness of the feedback. This is a problem that many software developers run into constantly. The people who can tell them if the result is correct aren’t always easily available to ask. I used to work with two very good business analysts. We’d run our software in test and sometimes we came up with outputs that we simply didn’t understand. The business analysts may have taken a few hours to get us an answer but they could always tell us if the output was correct or not.

Now think about AI generated software. I have the issue of how do I give the AI the right prompt to get what I want from it. And if I’m a non-developer I’ve got the even larger issue of evaluating the feedback it gives me. If I ask for C code to add 21 and 21 and I get this back from the AI:

1
2
3
4

int main()
{
    return 21 + 21;
}

I’m pretty confident that’s correct. But it’s also trivial and I know how to code in C. For a non-developer it might as well be Greek. And more importantly not having the correct background and experience how can they judge the feedback the AI gives them for correctness?